Pattern Analysis and Synthesis in Attractor Neural Networks

The representation of hidden variable models by attractor neural net works is studied Memories are stored in a dynamical attractor that is a continuous manifold of xed points as illustrated by linear and nonlinear networks with hidden neurons Pattern analysis and synthesis are forms of pattern completion by recall of a stored memory Analysis and synthesis in the linear network are performed by bottom up and top down connec tions In the nonlinear network the analysis computation additionally requires recti cation nonlinearity and inner product inhibition between hidden neurons One popular approach to sensory processing is based on generative models which assume that sensory input patterns are synthesized from some underlying hidden variables For example the sounds of speech can be synthesized from a sequence of phonemes and images of a face can be synthesized from pose and lighting variables Hidden variables are useful because they constitute a simpler representation of the variables that are visible in the sensory input Using a generative model for sensory processing requires a method of pattern analysis Given a sensory input pattern analysis is the recovery of the hidden variables from which it was synthesized In other words analysis and synthesis are inverses of each other There are a number of approaches to pattern analy sis In analysis by synthesis the synthetic model is embedded inside a negative feedback loop Another approach is to construct a separate analysis model This paper explores a third approach in which visible hidden pairs are em bedded as attractive xed points or attractors in the state space of a recurrent neural network The attractors can be regarded as memories stored in the net work and analysis and synthesis as forms of pattern completion by recall of a memory The approach is illustrated with linear and nonlinear network ar chitectures In both networks the synthetic model is linear as in principal